Section: New Software and Platforms

PapARt

Participant : Jérémy Laviole [Main developer] .

As part of his thesis work, Jérémy Laviole has developed a software suite for PapARt : Paper Augmented Reality Toolkit. This work is being extended to become a state-of-the-art library for projection mapping (spatial augmented reality) and tangible interfaces.

Papart is a Processing (http://www.processing.org ) library, and follows the Processing philosophy of openness and ease of use. The main features are :

1. Augmented reality rendering that enables rendering for cameras and projectors.

2. Tracking for Augmented Reality : tracking from ARToolkitPlus (https://launchpad.net/artoolkitplus ), and OpenCV SURF (http://opencv.org/ ). Extensions with other libraries are planned.

3. Camera support : in addition to the Processing Video library, PapARt support for video from OpenCV, OpenKinect, FFMPEG, FlyCapture and more is available from JavaCV (http://bytedeco.org/ ). It provides a wide support on Mac, Windows, Linux and possibly Android.

4. "Tactile" input on planar surface : Touch and hovering can be detected by a depth camera such as Kinect (http://www.microsoft.com/en-us/kinectforwindows/ ), the current support is Kinect XBOX 360 with openKinect drivers. Extensions are planned for Kinect 2, Kinect for Windows, Microsoft SDK (on Windows), DepthSense and most consumer market depth cameras. It scales elegantly from touch input from finger on small surfaces (such as A3 size) to large surfaces (2m x 3m size).

5. Software infrastructure to create "paper touch screens", following Processing's methods to create drawings and interactive experiences.

Technical challenges for the next few years :

• Color camera, depth camera and projector calibration made easy and more automated.

• Software and hardware installation of such cameras documented with tutorials and technical advices.

Research questions and challenges :

• Creation of tangible interfaces, tangibles elements can be tracked from cameras and depth cameras.

• Capture of part of pieces of paper for image analysis. E.g. to analyse and monitor drawings.

• Interactive projection mapping is an active research field, and such tools could power new research projects.

website: http://papart.gforge.inria.fr